Resource Scheduling Strategy for Spark in Co-allocated Data Centers

نویسندگان

چکیده

Abstract The co-allocated data centers are to deploy online services and offline workloads in the same cluster improve utilization of resources. Spark application is a typical batch workload. At present, resource scheduling strategy for mainly focuses on services. applications still use original scheduling, which can’t solve dependency deadline problems between spark This paper proposes data-aware resource-scheduling model meet requirement optimize throughput processing premise ensuring quality service

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Resource-Aware Scheduling for Data Centers with Heterogenous Servers

This paper presents an algorithm for resource-aware scheduling of computational jobs in a large-scale heterogeneous data center. The algorithm aims to allocate different machine configurations to job classes to attain an efficient mapping between job resource request profiles and machine resource capacity profiles. We propose a three-stage algorithm. The first stage uses a queueing model that t...

متن کامل

A New Job Scheduling in Data Grid Environment Based on Data and Computational Resource Availability

Data Grid is an infrastructure that controls huge amount of data files, and provides intensive computational resources across geographically distributed collaboration. The heterogeneity and geographic dispersion of grid resources and applications place some complex problems such as job scheduling. Most existing scheduling algorithms in Grids only focus on one kind of Grid jobs which can be data...

متن کامل

Effective Resource Management for Virtualized Data centers

Time and Technology has its significance made us to think and switching from classical to modern era of Information technology. If we look forward to the best of the Industrial world where Technology is the use of Information technology organization especially service based organization, provides a solution telling so and so high approaches. But we know the crackers and hackers may be ethical o...

متن کامل

Energy Aware Resource Management of Cloud Data Centers

Cloud Computing, the long-held dream of computing as a utility, has the potential to transform a large part of the IT industry, making software even more attractive as a service and shaping the way IT hardware is designed and purchased. Virtualization technology forms a key concept for new cloud computing architectures. The data centers are used to provide cloud services burdening a significant...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Lecture Notes in Electrical Engineering

سال: 2022

ISSN: ['1876-1100', '1876-1119']

DOI: https://doi.org/10.1007/978-981-19-2456-9_13